Common techniques to detect an obstacle typically use a laser or a supersonic wave, and a technique using a TV camera. The technique using the laser is typically expensive and not practical. The technique using the supersonic wave includes a problem for detection accuracy of the obstacle because resolution of the supersonic wave is usually low. Furthermore, in a single active sensor using the laser or the supersonic wave, a driving lane of a vehicle cannot be recognized.
On the other hand, the TV camera is relatively cheap and suitable for obstacle detection from a viewpoint of a resolution, instrumentation accuracy and an instrumentation area. Furthermore, the driving lane of the vehicle can be recognized. In the case of using the TV camera, there are a method using one camera and a method using a plurality of cameras (stereo camera). In the method using one camera, a road area and an obstacle area are separated from an image input through the one camera by using information such as intensity, a color, or a texture. For example, an area of intermediate intensity (brightness is low), e.g., a gray area, is extracted from the image as the road area. Otherwise, an area of few textures is extracted from the image as the road area. In this case, an area (except for the road area in the image) is regarded as an obstacle area.
The method using a plurality of cameras is called “a stereopsis”. In general, the stereopsis is a technique to calculate a three-dimensional coordinate of arbitrary point of the image in a coordinate system (hereafter, it is called a stereo camera coordinate) fixed to the stereo camera. The obstacle is detected using three-dimensional information obtained by the stereopsis.
In the stereopsis, for example, two cameras are located at the right and left sides, a point in three-dimensional space is corresponded between the right image and the left image, and a three-dimensional position of the point is calculated by using a triangulation. If a position and a posture of each camera for a road plane are previously measured, a height of an arbitrary point in the image from the road plane can be calculated by using the stereopsis. Briefly, the obstacle area is separated from the road area on the image by calculating the height. In the method using one camera, the obstacle object cannot be correctly separated from the road area because many obstacles of which intensity, color or texture is similar to actual road exist. In the stereopsis, such problem can be avoided.
However, normal stereopsis includes a problem, i.e., a search of corresponding points, which is one element to prevent a realization of the stereopsis. The search of corresponding points is calculation necessary to correspond the same point in space with the right and left images. The calculation quantity is extremely large and the processing speed is often slow.
A method for fastly detecting an obstacle on the road using the stereo camera without the search of corresponding points is disclosed in Japanese Patent Publications (Kokai) P2001-76128 and P2000-293693. In this method, a road surface is assumed as a plane and an image conversion equation T to correspond a pixel point of the road surface on one camera image (camera image 1) with a corresponding pixel of the road surface on another camera image (camera image 2) is determined by a geometrical relationship between the stereo camera and the road surface. The obstacle is detected by a difference between the camera image 2 and an image (conversion image) converted from the camera image 1 using the image conversion equation T. Briefly, each pixel of the road area on the camera image 1 is correctly converted to a corresponding pixel of the road area on the camera image 2 by image conversion T. On the other hand, each pixel of the object area (obstacle area) having a height on the camera image 1 is not correctly converted to a corresponding pixel on the camera image 2. Accordingly, the obstacle is fastly detected using a difference between the camera image 2 and the conversion image.
However, in this method, it often happens that the object is not correctly detected. For example, if an obstacle, a component object or a scene reflects on a road surface which is wet by rain, it is difficult to correctly detect the obstacle or the object on the road surface. A reflection on the road surface is virtually regarded as an object having a negative height (a virtual image). Each pixel of the virtual image is not correctly converted to a corresponding pixel on another image by above-mentioned conversion method. In the same way as the obstacle, the reflection on the road surface causes a difference between the camera image 2 and the conversion image. Briefly, it often happens that the obstacle is not correctly detected from the image because of an erroneous difference caused by the reflection.